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Abstract 

Quality problem. The new national patient-controlled electronic health record is an important quality improvement, and there 
was a pressing need to pilot its use in Australian primary care practices. Implementation of electronic health records in other 
countries has met with mixed success. 

Initial assessment. New work was required in general practices participating in the national electronic health record. National 
implementers needed to engage with small private general practices to test the changes before general introduction. 

Choice of solution. The National E-health Transition Authority' contracted the Improvement Foundation Australia to conduct a 
quality' improvement collaborative based on 9 years of experience with the Australian Primary Care Collaborative Program. 

Implementation. Aims, measures and change ideas were addressed in a collaborative programme of workshops and supported 
activity periods. Data quality measures and numbers of health summaries uploaded were collected monthly. Challenges such as 
the delay in implementation of the electronic health summary were met. 

Evaluation. Fifty-six practices participated. Nine hundred and twenty-nine patients registered to participate, and 650 shared 
health summaries were uploaded. Five hundred and nineteen patient views occurred. Four hundred and twenty-one plan/do/ 
study/ act cycles were submitted by participating practices. 

Lessons learned. The collaborative methodology was adapted for implementing innovation and proved useful for engaging with 
multiple small practices, facilitating low-risk testing of processes, sharing ideas among participants, development of clinical cham- 
pions and development of resources to support wider use. Email discussion between participants and system designers facilitated 
improvements. Data quality was a key challenge for this innovation, and quality measures chosen require development. Patient 
participants were partners in improvement. 
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Quality problem 

New work is required in Australian family practices for creat- 
ing and uploading Shared Health Summaries (SHSs) to the 
new national, patient-controlled electronic health record. How 
could the new processes be tested in real-life, busy primary 
care practices before roll-out across the country? 

The SHS comprises a patient's medical history, their medi- 
cation list, their allergies and their immunization record. An 



SHS is uploaded and updated by a particular provider nomi- 
nated by the patient (usually their general practitioner/ family 
physician). Patients can review their summary before it is 
uploaded to the national record. 

Efforts to improve the quality of health care through elec- 
tronic health records have had mixed results with proof of 
benefit hard to find [1, 2]. One large systematic review of the 
impact of health information technology found benefit in 
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adherence to guidelines, in surveillance and monitoring and in 
decreased medication error [2]. Veterans Affairs (USA) [3] and 
Kaiser Permanente [4] have claimed positive cost benefit from 
their ehealth record systems. The NHS England implementa- 
tion had mixed success with the Summary Care Record being 
retained and having strong growth from 5 million records in 
January 2011 to over 34 million records by February 2014 [5]. 

Patients favour accessing their own records [6—11]. There 
is evidence that patient access to records enhances the 
doctor— patient relationship [9], increases adherence [7, 8] and 
increases empowerment [11], though up to a third of patients 
have difficulty understanding their own records [8]. 

Initial assessment 

By November 2011, the imminent date for the electronic 
health record infrastructure being launched (July 2012) made 
testing the new processes in actual primary care practices a 
pressing priority. The National E-health Transition Authority 
(NEHTA) faced the challenge of doing this with the many 
small, privately owned businesses that provide most primary 
care in Australia. 

The new tools, skills, processes and systems needed to 
upload the SHS include the following: 

• processes to register patients, providers and organizations 
in the online system, 

• updating computers and software to be compliant with 
the new system, 

• processes to improve and maintain data quality to ensure 
SHS are useful, 

• processes for upload of summaries and 

• obtaining patient consent and engaging them in ensuring 
the accuracy of clinical information. 

Choice of solution 

NEHTA contracted the Improvement Foundation (IF) to con- 
duct a quality improvement collaborative (QIC) to pilot these 
innovations, test systems and technology, identify improvements 
required and provide adoption stories and champions. 

QICs have been used extensively to improve health systems. 
Evaluations have identified the QIC as possibly the most ef- 
fective strategy available for closing the 'quality chasm' while 
acknowledging unanswered questions about cost effectiveness 
[12-14]. 

The Australian Primary Care Collaboratives (APCC) 
Program has employed QICs to improve primary care since 
2005 [15—17]. The success of this programme in improving 
chronic disease care led to it being chosen for this project. 

Implementation 

The collaborative occurred between November 2011 and 
August 2012. A reference panel made up of information tech- 
nology experts, general practice experts and improvement 
experts set the aim (Box 1), the change principles (Box 2) and 



Box I . Aim of the collaborative programme 

Two thousand patients from participating general practices 
and health services in the eCollaborative wave have an 
accurate SHS established and maintained on a PCEHR by 
1 August 2012. 



the improvement measures (Box 3) for the project. This was 
done in a 1-day meeting designed to achieve realistic com- 
promise between theoretical and practical priorities. The aim 
was deliberately set as a stretch that would be hard for the 
group to achieve. Measures addressed the number of SHS 
uploaded as well as practice data quality. Data quality measures 
needed to be extractable with existing tools used by the IE 
These measures were brainstormed by the group and nar- 
rowed to two, which were thought likely to reflect improve- 
ments in the medication list and past history list, which are two 
important fields in the SHS. 

The quality of past history recording was assessed by meas- 
uring the percentage of diagnoses, which had been recorded 
using a recognized coding system. The quality of the medica- 
tion list was measured by assessing its currency. This was done 
by measuring the percentage of medications on the list for 
which a current script had been issued. 

The change ideas listed below were proposed by the Expert 
Reference Panel after considering the processes required for 
creating and uploading an SHS. As this was a new activity, it 
was expected that they would be modified and new ideas 
created by the participating practices to suit their local teams 
and circumstances. 

Small practice teams completed a series of three webinars 
alternating with two face-to-face national workshops (see 
Fig. 1). Between workshops, teams returned to their practices 
for activity periods implementing the change principles they 
had learnt. Practices used plan/do/study/act (PDSA) cycles 
to test and refine the changes that they made. 

Each month, the data quality measures were extracted by 
practices from their practice software and uploaded to the 
qiConnect data portal of the IF. A monthly count was made of 
uploaded SHS. Results were fed back to practices to show pro- 
gress and allow comparison with the group. 

Sharing of successful strategies and failures occurred at 
workshops, through a wiki, through an email mailing list and 
at weekly 'Tweet Up's' using Twitter. A series of audio and 
video podcasts and online instruction guides were produced. 

Practices were supported by Collaborative Program 
Managers (CPMs) situated in their regional primary health 
care organization. A small experienced team from the IF orga- 
nized the events, coached and supported the CPMs, coordi- 
nated data management and solved technical problems with 
the help of key NEHTA staff. 

Progress in the QIC was assessed by changes in improve- 
ment measures. Participation was assessed by data submission 
rates, attendance at workshops, attendance at webinars and 
social media activity. Participants were required to submit 
PDSA cycles online as an indication of improvement activity. 
Each workshop was evaluated by participants. 
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Box 2. eCollaborative change principles and change principles 
Change principle Change ideas 


Build and maintain the 
team 


Set realistic goals 

Communicate with other team members 

Engage the team 

Assign roles and responsibilities 

Reflect on and review what you are doing 


Undertake foundational 
work for the electronic 
health record 


Obtain a health provider identifier for individual health professionals (e.g. general practitioners, 

registered nurses and aboriginal health workers) (HPI-I) 

Obtain a health provider identifier for your organization (HPI-O) 

Obtain a patient's individual health identifier 


Develop systems to 
improve and maintain 
data quality across your 
clinical system 


Archive inactive and deceased patients 
Use consistent disease coding 

Develop a practice policy to ensure that patient's current problem and past history diagnoses are 
reviewed regularly 

Record results in the right place, including pathology results with HL7 

Use your clinical software or compatible data extraction tool to improve data quality 

Develop systems to maintain data quality, including medications, allergies and immunizations lists 


Develop systems to 
support the SHS 


Use process mapping to identify opportunities to integrate the SHS into the general practice and 
health service work flow 

Decide and document the nominated provider(s) responsible for maintaining the currency and 
accuracy of the SHS 

Inform your practice team about the processes of uploading an SHS 

Develop policy and procedures for checking, uploading and maintaining an SHS 


Engage your patients in 
the electronic health 
record system 


Identify patients who would most benefit from an SHS 

Inform your patients about the electronic health record system and, in particular, the SHS 
Develop systems to obtain informed consent from identified patients 
Process map the points of care at which patients can input into their SHS 
Design processes to increase patient review of their SHS 

Develop policies about points of care when patients are prompted to review their SHS 

Develop processes for patients to advise of any changes to their medication lists 

Seek patient advice on how they can contribute to the development and maintenance of their SHS 


Develop systems to 
improve the integration 
of care through use of 
the SHS across the care 
team 


Communicate with your local providers concerning the SHS 

Develop ways to use the SHS with other members of the patient's care team 

Develop processes that include the SHS in team care arrangements, Home Medicine Reviews, 

mental health plans, referrals to emergency, specialists and allied health providers 

Explore tools and resources to share care plans electronically 

Engage the patient to develop new strategies for sharing their SHS across their care team. 


Develop systems and 
processes to improve 
patient self-management 
skills using the electronic 
health record system. 


Use the electronic health record system to improve health literacy and self-efficacy 

Ensure the whole team is aware of and able to access the electronic health record to enhance 

patient self-management skills 

Develop key messages and terminology that support the reinforcement of key self-management 
concepts by the team 

Explore patients' use of the electronic health record 

Use the electronic health record system to promote other online health resources 

Bring patients together to provide feedback about the electronic health record and how its use can 

be improved 
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Evaluation 

Practices were keen to participate with 222 expressions of 
interest received between 18th and 25th November 2011. 
Funding permitted 56 practices to begin the QIC. Two with- 
drew leaving 54 completing. 

As the national infrastructure for uploading of SHSs was 
delayed, two alternative software solutions were used to enable 
participants to test the practice processes required. The delay 
in provision of the national technological solution as well as 
the temporary nature of the alternative structures delayed and 
reduced the number of summaries uploaded. 

Participants communicated with each other, IF and 
NEHTA staff at national workshops, through their CPMs, by 
email and through social media. Email discussion was particu- 
larly active and detailed. NEHTA staff monitored the email 
discussion and gave rapid responses to the challenges practice 
staff were experiencing. Problems in registration of providers 
and consumers, the upload process and other areas were pro- 
gressively identified and addressed. Issues were recorded and 
passed on to the appropriate party such as NEHTA, and other 
government departments and agencies. These key participants 
emerged as clinical leaders of the Collaborative and later were 
local champions for wider implementation. 

A GP with a special interest was appointed social media dir- 
ector of the eCollaborative in order to explore and promote its 
use. The eCollaborative Twitter account had 93 registered fol- 
lowers with 217 'tweets' being shared. Only 5 of the 54 partici- 
pating practices actively contributed to the Tweet ups. A larger 
number listened in including government and media, which 
led to some external attention [18]. One workshop had live 



Box 3. eCollaborative improvement measures 

Main outcome measure 

Number of SHSs uploaded 
Data quality measures 

Percentage of diagnoses in the clinical database that are 
coded 

Percentage of prescriptions in the 'active medication list' 
that are current 



tweets displayed allowing participants to comment throughout 
the day. Feedback indicated that Twitter may have been utilized 
more frequently if participants were provided with detailed in- 
struction on its functionality and advantages. 

A wiki to which all participants could contribute was devel- 
oped for the eCollaborative wave as a central repository of 
news and resources. A survey of participants in the collabora- 
tive indicated that 75% had a preference for face-to-face work- 
shops and 74% chose email as their preferred method of 
communication. 

Patients participated in workshops and provided advice and 
enthusiasm for the project. Concrete suggestions from them 
included using patients to recruit other patients to register for the 
PCEHR and using an iPad to share their summary with their 
specialist. For the trial system used in this project, patients and 
providers were given a secure password to access the patient's 
SHS. Patients gave informed consent to have their SHS uploaded. 
They expressed significant desire for rapid implementation of 
the National eHealth Record System. Practices were funded to 
include patient input in their local improvement changes. 

Practice electronic health record data quality improvement 
measures are reported in Figs 2 and 3 and show little change. 
It is possible that the measures failed to capture the improve- 
ment that did occur. This is supported by the observation that 
there was a considerable increase in the absolute numbers of 
current scripts and coded diagnoses, suggesting that work was 
done and improvements were made. The development of useful, 
validated data quality improvement measures for Australian 
general practice is a need demonstrated by this project. 

Nine hundred and twenty-nine patients registered to partici- 
pate in the project. Six hundred and fifty SHSs were submitted 
as indicated in Fig. 4. There were 519 patient 'views' of 
records. Four hundred and twenty-one PDSA cycles were sub- 
mitted, which is an average of 7.6 per participating practice. 



Lessons learnt 

This collaborative was different from the many health im- 
provement collaboratives that the APCC has conducted, in 
that it focussed on a technical innovation rather than 
known health evidence. The Breakthrough Series quality 




Figure I Outline of collaborative programme. 
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Figure 3 Average percentage of scripts current in practice electronic health records (« = 56). 



improvement collaborative methodology was created to 'help 
health care organizations make "breakthrough" improve- 
ments in quality' [19]. This collaborative is an adaptation of 
the methodology to test a technological innovation. Primary 
care-focussed health systems are emerging as the key to high- 
quality, cost-effective provision of health care in the future, 
but spreading innovation in multiple small private practices 
can be difficult. 

Experience overseas has shown that introducing a web-based 
electronic health record is not easy, and other countries have 
been variably successful. The upload of 650 summaries in real- 
world general practices under a tight time frame was a significant 
achievement while falling short of the stretch goal of 2000. Nine 
hundred and twenty-nine patient registrations and 519 views in- 
dicate good consumer engagement with the project. 

Other health systems seeking to innovate in a similar 
primary care context may learn from the lessons listed below 



We found the QIC to be a useful methodology for 
testing innovations because 

• The collaborative methodology provided a known and 
trusted model for engagement with multiple small private 
practices. Ecollaborative practices had previously partici- 
pated in the APCC and so understood the methodology 
and quality improvement principles. The collaborative 
format has proven to be effective for building enthusiasm 
in participants, which was particularly important when 
dealing with a new topic such as ehealth changes. 

• The existing infrastructure and methodology for collect- 
ing and feeding back data developed for the APCC could 
be easily adapted to support this project. 

• Using a model such as a QIC limits risk by testing new 
technology with a small number of real-world practices. 
The QIC strategy is one model for trialling processes in a 
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small number of well-disposed, resilient real-world prac- 
tices prior to wider roll-out and deserves consideration by 
others undertaking similar projects. 

• By closely connecting the implem enters (NEHTA) and the 
frontline clinicians, the QIC model used in this project 
facilitated identification and correction of difficulties with 
the technology of the innovation. Group email discussion 
proved the best channel for this. While interim systems 
were used in this collaborative, the processes involved in 
uploading were a good reflection of the mature system. 
NEHTA used an explicit process to capture and spread 
the lessons learnt by participating practices. 

• QICs provide a context for the development of cham- 
pions. Enthusiastic and creative individuals emerged as 
the collaborative progressed who participated in solving 
problems. Some of these have become credible clinical 
champions for the introduction of the national electronic 
health record in their local regions. 

• It facilitated development and testing of resources and 
frequently asked questions. Development of these prior 
to the public launching of the electronic health record 
has improved credibility and smoothed wider imple- 
mentation. Lessons from the collaborative have been 
incorporated into the Australian eHealth learning centre 
(http : // www. ehealth .gov. au) . 

Data quality remains a challenge after technical 
implementation has been achieved 

Good clinical data quality will require constant effort by all 
practices if useful information is to be uploaded and kept 
current to improve patient care. 

Reliable, useful measures of data quality are required so that 
practices can measure the effectiveness of their improvement 
efforts in this area. Some participating practices were able to 
achieve large improvements in the data quality measures; 
however, overall the measures we chose were not responsive to 
the data improvement work done by the practices. In the short 
period of the collaborative practices tended to focus on 



improving the data of patients whom they considered would 
most benefit from having an SHS such as those with multiple 
chronic conditions and older patients. Improving data of these 
patients did not result in noticeable movement in the overall 
data quality of the patient population of the practices. 
Measures of data quality are being reviewed. It may be better 
to measure changes in data quality in subgroups. 

Patients can be partners in implementation 

Patients were enthusiastic about the electronic health record. 
They were creative in identifying strategies for registering other 
patients in the ehealth system and for using the records to 
inform their health care team. Having patient participation in 
workshops helped clinicians see beyond their own personal and 
organizational challenges to the importance of the innovation for 
patient care. Patients proved effective allies in checking the accur- 
acy of their own clinical data and helping practices improve it. 
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